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29 May 1980 


MEMORANDUM FOR: ———————— 
cting Virector, Program Assessment Office, RMS 


FROM: 


SUBJECT: 


1. 


Deputy to the DCI for Resource Management 


[__]Study Report 


I have the read the[_ |study over fairly carefully and have a 


number of comments on it. I will defer comments on the statistical 
issue until after our discussion. 


eo. 


Your covering memo cited a pair of key judgments which were 


deleted from the report. I do want to explore with you how we could 
implement the suggestion to require quantitative and qualitative product 
evaluation data with requests for collection system upgrades. I also 
want to explore how we can get NFAC, DIA, and INR to investigate how 
finished intelligence might better respond. You were correct in deleting 
these findings, however. 


3. 


My other comments are orqan: zed by page. 


~- Page 4. This is the first occurence of reference to the 
small sample size. This issue is fuzzy throughout. Remember 
that the relationship between a sample and the size of the 
universe from which it ‘s drawn has only a very minor affect 
on statistical significance. That is why opinion polls 
drawing on a sample of perhaps 1200 people out of 220 million 
can have statistical va'idity. The more important question 
is the relationship between the sample size and the complexity 
of the questions being posed to the data. Beyond that, there 
is the perhaps more fundamental question of the way the sample 
ts drawn and whether one gan appeal to arguments resting on 
randomness. For tHé‘@firas measures--like the Proportion of 
messages that relate to po}itical or economic matters, the 
sample may very well be sufficien re fine-grained 
issues--the value of a particular] | as the paper 
rightly argues, the sample is probably Insufficient to 
Support many conclusions. 
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~~ Page 8 and following pages. All of the tables need to have 
feadings which describe more fully what the percentages mean. 
In other words, what variable is the indicated percentage of 


what other variable¢ 


when they rated an itam as “essential” or "important." What 
questions were policymakers responding to? To what activity 
was the report in question deemed to be essential? 


~~ Page 22. Can you explain to me what is? I do not STAT 
understand what is the 25X1 


-- Page 29. I would lik to know more about [Jin NSA. STAT 
STAT 
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